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The National Household Education Survey (NHES) is a data 
collection system of the National Center for Education Statistics, which has 
legislative mission of collecting and publishing data on the condition of 
education in the United States. The NHES provides information on educational 
issues that are best addressed by contacting households rather than 
educational institutions. It is a telephone survey of the 

noninstitutionalized civilian population of the United States. As part of a 
methodological study, the NHES for 1991 contained an experiment to test the 
feasibility of using the NHES as a mechanism to conduct longitudinal studies 
of young children. This report describes the design, procedures, and results 
of the followup survey. In NHES: 91, parents of children aged 3 to 8 years 
were surveyed about the care and education of their children for the NHES 
Early Childhood Education component. The feasibility of a longitudinal 
followup survey was tested by drawing a random subsample of children from 
that survey and attempting to locate the same respondents about 1 year later. 
A sample of 513 children was selected, and of the 452 parents or guardians 
located in the followup, 90% were successfully interviewed at the later date. 
Implications for longitudinal studies are discussed. It appears that the 
longitudinal response rate for such a survey would probably be about 62 to 
65%. (Contains 8 tables, 1 figure, 1 exhibit, and 22 references.) (SLD) 
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Background of the NHES 

Most large random-digit-dial telephone surveys 
conducted in the United States are cross-sectional 
studies that examine characteristics of the 
population at a particular point in time. A 
longitudinal design is another method for 
surveying the population that has important 
advantages over repeated cross-sectional surveys 
for estimating changes over time. This report 
examines the feasibility of conducting a 
longitudinal survey when the base year data are 
from a random-digit-dial telephone survey. The 
National Household Education Survey was the 
vehicle for this research and is described below 
prior to a more complete discussion of the rationale 
for the study. 

The National Household Education Survey 
(NHES) is a data collection system of the National 
Center for Education Statistics (NCES), which has 
as its legislative mission the collection and 
publication of data on the condition of education in 
the Nation. The NHES is specifically designed to 
support this mission by providing information on 
those educational issues that are best addressed by 
contacting households rather than schools or other 
educational institutions. The NHES provides 
descriptive data on the educational activities of the 
U.S. population and offers policymakers, 
researchers, and educators a variety of statistics on 
the condition of education in the United States. 

The NHES is a telephone survey of the 
noninstitutionalized civilian population of the 
United States. Households are selected for the 
survey using random-digit-dialing (RDD) methods, 
and data are collected using computer-assisted 
telephone interviewing (CATI) procedures. 
Approximately 60,000 households are screened for 
each administration, and individuals within 
households who meet predetermined criteria are 
sampled for more detailed or extended interviews. 
The data are weighted to permit estimates of the 
entire population. The NHES survey for a given 
year typically consists of a Screener, which collects 
household composition and demographic data, and 
extended interviews on two substantive 
components addressing education-related topics. 



In order to assess data item reliability and inform 
future NHES surveys, each administration also 
includes a subsample of respondents for a 
reinterview. 

The primary purpose of the NHES is to conduct 
repeated measurements of the same phenomena at 
different points in time. Throughout its history, the 
NHES has collected data in ways that permit 
estimates to be tracked across time. This includes 
repeating topical components on a rotating basis in 
order to provide comparative data across survey 
years. In addition, each administration of the 
NHES has benefitted from experiences with 
previous cycles, resulting in enhancements to the 
survey procedures and content. Thus, while the 
survey affords the opportunity for tracking 
phenomena across time, it is also dynamic in 
addressing new issues and including conceptual 
and methodological refinements. 

A new design feature of the NHES program 
implemented in the NHES:96 was the collection of 
demographic and educational information on 
members of all screened households, rather than 
just those households potentially eligible for a 
topical component. In addition, this expanded 
screening feature included a brief set of questions 
on an issue of interest to education program 
administrators or policymakers. The total Screener 
sample size was sufficient to produce state 
estimates of household characteristics for the 
NHES:96. 

Full-scale implementations of the NHES have been 
conducted in 1991, 1993, 1995, and 1996. Topics 
addressed by the NHES:91 were early childhood 
education and adult education. The NHES:93 
collected information about school readiness and 
school safety and discipline. The 1991 
components were repeated for the NHES:95, 
addressing early childhood program participation 
and adult education. Both components underwent 
substantial redesign to incorporate new issues and 
develop new measurement approaches. In the 
NHES:96, the topical components were 
parent/family involvement in education and civic 
involvement. The NHES:96 expanded screening 



feature included a set of questions on public library 
use. 

In addition to its topical components, the NHES 
system has also included a number of 

methodological investigations. These have 
resulted in technical reports and working papers 
covering diverse topics such as telephone 
undercoverage bias, proxy reporting, and sampling 
methods. This series of technical reports and 
working papers provides valuable information on 
ways of improving the NHES. 

Rationale for a Longitudinal 
Feasibility Study 

The NHES employs a cross-sectional design, 
which allows for the collection of detailed 
information on independent samples of households 
that are different across administrations. In 
contrast, studies using a longitudinal design collect 
data from a sample of respondents at more than 
one point in time. The ability to track a sample 
across time, addressing the same research 
questions, can provide precise measurement of the 
process and outcomes of educational experiences. 

Longitudinal studies are often undertaken to study 
how experiences and activities relate to outcomes 
that occur later in time. For example, longitudinal 
studies of students such as High School and 
Beyond (Zahs et al. 1995), the National 
Educational Longitudinal Study (Ingels 1994), and 
the Beginning Postsecondary Students 
Longitudinal Study (Fitzgerald 1994) have been 
used to measure relationships between the 
experiences of students and later educational 
achievement, enrollment in postsecondary 
schooling, and career paths. For longitudinal 
studies of young children, a primary objective is to 
better understand how early childhood experiences, 
including participation in educational programs, 
prepare children for success in school. 

Longitudinal data collection programs also provide 
a unique opportunity to examine the process of 
change. For example, with a longitudinal approach 
to early childhood experiences, it is possible to 



accurately capture the number and types of 
programs in which a child participates at different 
points in his or her early years, as well as to 
examine how the types of programs change in 
relation to changing family composition and 
economic circumstances. Such data may be of 
great value in assessing how these early 
experiences are associated with later educational 
progress. 

There are other methodological features associated 
with using longitudinal designs in educational 
research (Bailar 1989). For example, cross- 
sectional surveys have attempted to assess change 
in children's development over time by collecting 
retrospective information from parents. However, 
reliance on respondents' memory can be 
particularly prone to error. A longitudinal 
approach reduces the error associated with recall 
because the data are collected near the time the 
events occur. The accuracy afforded by this 
approach may permit the identification of 
relationships that might be obscured by the 
response errors inherent in cross-sectional surveys. 

Since one of the goals of the NHES is to monitor 
changes over time, both nationally and for key 
subgroups, longitudinal approaches have a natural 
appeal. Duncan and Kalton (1987) describe a 
variety of measures of change and the design 
approaches that are most advantageous for these 
measures. A longitudinal design is the only 

approach that permits estimates of changes at the 
individual (micro) level. For example, a 

longitudinal design is necessary to examine how 
events and experiences in a child's early life affect 
performance later in life, assuming retrospective 
data collection is not deemed reliable. The 
implementation of a longitudinal design would 
greatly enhance the analytic potential of the NHES 
data sets. Assessing the ability of the NHES to be 
a vehicle to address these types of issues in 
addition to providing cross-sectional estimates of 
the population was the motivation for this study. 

While much is known about the difficulties and 
successes associated with longitudinal studies 
based on institutional samples or in-person base 
year interviews, little is known about longitudinal 
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studies based on random-digit-dial (RDD) samples. 
Booth and Johnson (1985) report on a study of just 
over 2,000 married couples who were selected by 
RDD in the base year and were interviewed 3 years 
later, after interim tracking contacts. 

As a part of a methodological investigation, the 
NHES:91 contained an experiment to test the 
feasibility of using the NHES as a mechanism to 
conduct longitudinal studies of young children. 
The complex design and methodological 
procedures of the NHES present important 
challenges to undertaking a longitudinal approach. 
For example, random-digit-dial surveys typically 
have lower response rates than personal interview 
surveys, with much of the loss in response 
occurring within the first moments of contact with 
the household (Groves and Lyberg 1988). If 
tracing RDD samples is also problematic, 
substantial additional numbers of cases could be 
lost to the study. The net response rates that can be 
obtained in a longitudinal study must be evaluated 
in light of this fact. Booth and Johnson (1985) for 
example, report a base year response rate of 65 
percent and a followup completion rate of 78 
percent, for a time 2 response rate of 5 1 percent. 
The NHES feasibility study focuses on the issue of 
response rates but also addresses other challenges 
associated with a longitudinal design. 

The subsequent sections of the report describe the 
design, procedures, and results of the followup 
survey. The final section summarizes the research 
and discusses other methods that may be used to 
improve the estimates of change over time from the 
NHES. 



Design of the Followup Survey 

In the NHES:91, parents of 3- to 8-year-old 
children were surveyed about the care and 
education of their children for the Early Childhood 
Education (ECE) component. All eligible children 
were sampled within each of the households for 
which a screening interview was completed. While 
the child is the unit of analysis, the parent or 
guardian who was most knowledgeable about the 
child’s care and education was the respondent to 



the ECE interview. In all, 13,894 interviews were 
completed in the ECE component. These 
completed interviews represented a net response 
rate of 77 percent, the product of the screening 
response rate of 81 percent and the extended 
interview (ECE) response rate of 95 percent. Brick 
et al. (1992b) presents additional information on 
the NHES:91 ECE component design, survey 
procedures, and unit and item response. 

As noted earlier, conducting a longitudinal 
followup survey with the respondents to the NHES 
has some analytic advantages, but the feasibility of 
successfully accomplishing this data collection had 
not been evaluated prior to this experiment. The 
experiment involved drawing a random subsample 
of children who were the subjects of completed 
NHES:91 ECE interviews, asking the parent 
respondents a series of additional questions that 
would assist in locating them at a later date, 
attempting to locate the same respondents about 1 
year later, and conducting a brief followup 
interview with the respondents to determine how 
willing they would be to complete another 
interview. 

The subsample for the followup survey was 
selected during the original interview in 1991. 
When the household was sampled in 1991 for the 
ECE component, a subsample of children was 
selected to participate in the followup study; 513 
NHES:91 interviews were completed for children 
selected for the followup study. A sample of about 
500 is sufficient to estimate a response rate to the 
followup with a standard error of less than 2 
percent. All 13,894 children whose parents 
completed initial interviews were eligible for the 
followup study, with the exception of one 
constraint; the subsample was restricted so that no 
more than one child per household could be 
sampled for the followup. Because telephone 
interviews can be time-consuming, particularly for 
busy parents, limiting followup to a single child in 
each household was used to reduce the time 
commitment for parents and to contribute to a high 
followup response rate. 

Since the NHES:91 sample was selected using 
RDD methods, the only identifying information for 
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the sampled persons available at the time of the 
initial interviews was their telephone number. 
Additional data items were asked in the NHES:91 
to facilitate locating the sampled persons at a later 
time. These items included the respondent's full 
name and address and the name, address, telephone 
number, and relationship of a contact person 
outside the respondent's home who could be 
expected to know how to locate the person in the 
future. Nearly 90 percent of the respondents 
provided their own full names and addresses. 
Booth and Johnson (1985) report similar levels of 
response to items asking respondent information; 
89 percent of their respondents provided their 
names and addresses. However, only about two- 
thirds of the NHES respondents were willing to 
give the requested data about a contact person 
(Booth and Johnson report 72 percent cooperation 
in naming contact persons). A series of survey 
procedures was undertaken to locate and interview 
the sampled parents 1 year later. These procedures 
are described below. 



Survey Procedures for the Followup 

The followup survey began in late April 1992 and 
ended a month later, about 1 year after the initial 
NHES:91 data collection period. The survey 
procedures for a typical RDD survey were not 
appropriate for the followup survey because 
specific respondents to the NHES:91 were sought. 
Because those respondents could have changed 
telephone numbers and addresses during the year, 
procedures were developed to attempt to locate and 
then interview the NHES:91 respondents for the 
followup survey. In the followup interview, each 
respondent was asked to answer a limited subset of 
items about the sampled child. 

Figure 1 presents an overview of the locating and 
interviewing steps for the followup survey. Every 
sampled respondent was classified as having been 
located or not, depending on whether or not it was 
possible to verify that the respondent lived in a 
household that was called during the followup 
survey. For all the respondents who were located, 
an additional status code was recorded indicating 
whether or not an interview was completed. The 



two classifications (located and interviewed) are 
key for evaluating the potential of conducting 
meaningful followup surveys for future NHES 
administrations. 

Locating Procedures 

The entire subsample of 513 persons was included 
in the followup survey, regardless of whether the 
contact information was available from the 
NHES:91 interview. That is, those persons who 
provided additional followup information were 
included, as were persons who did not provide this 
information. For all 513 respondents to the 
NHES:91, at least the first name of the parent 
respondent was known. Most of the respondents 
(90 percent) gave their last name and address when 
these questions were asked in the supplemental set 
of items included in the NHES:91. Because the 
full name and address were needed for some 
locating procedures, the results of the locating 
activities are sometimes given separately for those 
who provided contact information and for those 
who did not. 

Prior to making any telephone calls to the 
respondents, postcards were mailed to all those 
who provided their name and address in the 
NHES:91 interview. The purposes of the postcard 
were to remind the respondents that they would be 
interviewed again shortly and to obtain address 
updates from the Postal Service for any persons 
who might have moved during the last year. 
Exhibit 1 is a copy of the postcard. The 
information from the postcards was used only 
when the sampled person no longer lived at the 
same telephone number and had to be located. 

The first attempt to contact all 513 persons for the 
interview was to call the telephone number used 
for the NHES:91 interview. Given that only 1 year 
had passed since the NHES:91 interview, a 
relatively large proportion of the respondents was 
expected to be located in a household with the 
same telephone number. Depending on the 
outcome of that call and the type of available 
information on the respondent, one of several 
possible locating efforts was followed. Telephone 




4 



10 



Figure 1. — Process of locating and interviewing respondents in the NHES:91 followup survey 




numbers of followup respondents were identified 
with the aid of five different sources, utilized as 
necessary in the following sequence: the NHES:91 
telephone numbers, recorded messages with new 
telephone numbers if the 1991 number was no 
longer in service at followup, directory assistance, 
the contact people provided in the NHES:91 
interview, and a credit reporting service. 

New technologies for locating persons using the 
Internet or CD-ROM files of phone listings may 
now be available to enhance locating efforts, but 
these methods were not in use at the time of this 
study. While these new technologies do offer a 
great advance, there are limitations associated with 
how current the data are and the percentage of 
people covered by the services. Continued 



improvements in technology promise even more 
effective and accessible services in the future. 

While many of the steps for locating respondents 
are self-explanatory, others require more detailed 
discussion. One of these is the use of directory 
assistance. In general, directory assistance 
telephone calls were made only for those 
respondents who gave their full name and address 
when interviewed in the NHES:91. The address 
used for the directory assistance calls was either 
the response from the NHES:91 interview or any 
update of that address obtained from the postcard 
mailing. Directory assistance was also attempted 
in a few cases where no address data existed, 
provided the last name of the respondent was 



uncommon enough that there was some chance of 
locating the appropriate respondent. 

Another locating step that requires further 
clarification is the use of the contact person from 
the NHES:91 interview. When a call to directory 
assistance did not yield a new telephone number 
for the respondent, the contact person identified in 
the NHES:91 supplemental items was called to 
obtain a new number for the respondent. This was 
done at the same time as the directory assistance 
telephone call. 

The last source for locating respondents was a 
credit reporting service. The full name and address 
of the respondent was required by the credit service 
to do this search, so those who did not give these 
data were excluded from this step. 

Schedule and Interviewing 

Interviewers who worked on the NHES:91 were 
assigned to conduct the followup interviews. Since 



they were already trained on the NHES procedures, 
this enabled us to reduce the amount of training 
needed for the followup. Six experienced 
interviewers received additional training for this 
work. 

A 4-hour training program included the following 
activities: 

■ a review of the NHES:91 ECE component and 
the followup feasibility test; 

■ procedures for conducting the followup; 

■ exercises showing each type of followup and the 
sequence of activities; and 

■ an interactive lecture to reacquaint the 
interviewers with the questions. 

The followup survey was done at Westat’s 
Telephone Research Center in Frederick, 
Maryland. The schedule of activities was as 
follows: 



Exhibit 1. — Postcard mailed to.NHES:91 followup sample 



National Household Education Survey 



About 1 year ago, you or a member of your household participated in a telephone 
interview for a Department of Education study on the care and education of young 
children. During that interview, we told you that we wanted to contact you again 
in about a year to ask a few followup questions about your child. We will be 
making these telephone calls in the next several weeks. As in the original 
interview, your participation is voluntary and confidential. This brief followup 
interview will only take about 5 minutes of your time. 

If you have any questions, please contact Kathryn Chandler at the U.S. 
Department of Education, National Center for Education Statistics, at 202-219- 
1767. You may also contact Mary Collins of Westat, Inc., who is managing the 
data collection effort, toll-free at 1-800-937-8281. 

Thank you for your continued assistance. 

OMB 1850-0640 
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Date Activity 

April 8, 1992 Postcards mailed to sample 

April 22, 1992 Training of interviewers 

April 22 — May 31,1 992 Interviewing 

May 8 — May 31,1 992 Tracing through credit 

service 

Some activities were done continuously during the 
data collection time period. These included 
updating addresses from postcard returns and 
making directory assistance calls as needed. 

When a telephone number was identified for a 
respondent, the number was dialed at least seven 
times over a 1 0-day period to attempt to locate the 
sampled person. If no contact with a household 
member was made during these repeated calls, an 
attempt was made to identify a new telephone 
number for the NHES:91 respondent until available 
numbers from all sources were exhausted. If the 
NHES:91 respondent was not found, then the case 
was classified as not located. 

If contact was made with a member of a household 
and he or she verified that the NHES:91 respondent 
or child was at this number, then the sampled child 
was classified as located. Multiple attempts were 
then made to interview the NHES:91 respondent (if 
the NHES:91 respondent was no longer a 
household member, but the child was, another 
responsible parent or guardian was interviewed). If 
after at least seven attempts an interview was not 
completed, the case was classified as located but 
not interviewed. 



Findings and Analysis 

In this section, we report on the findings from the 
followup efforts. The findings describe the overall 
success in locating and interviewing the NHES:91 
respondents. The results of specific efforts to 
locate the persons also are presented. The response 
experience for this feasibility study is placed in 
context with a review of longitudinal response rates 
from two other national household- 



based surveys. Each of these is an in-person 
survey, unlike the NHES, which uses a random- 
digit-dialing approach. Later, we discuss the 
implications of these results for conducting a 
longitudinal survey of children with the RDD 
methodology. 

Locating and Response Rates 

There were 513 persons eligible for the followup 
survey — the total number of persons who 
completed the ECE interview in the NHES:91 and 
were sampled for the followup. As a result of all 
the efforts combined, 88 percent of the sampled 
NHES:91 cases were located and 80 percent 
completed followup interviews. Of the 452 
NHES:91 parent/guardian respondents who were 
located in the followup, 90 percent were 
successfully interviewed (table 1). These rates are 
similar to those reported by Booth and Johnson 
(1985), who located 87 percent of their base year 
respondents, interviewed 90 percent of those who 
were located, and interviewed 78 percent of their 
total panel. Burgess (1989) reports that at least 80 
percent of survey respondents can be contacted for 
most surveys if tracing is used, and asserts that 
success rates in excess of 90 percent should be 
expected; the survey method associated with these 
rates (telephone or in person) is not specified. 

Table 1 shows the number and percent of 
respondents by response status, and for those who 
were located, the number and percent for whom 
interviews were completed. About half of the 
cases where the respondent was located but no 
interview was conducted were refusals, and almost 
all of the others were cases for which the 
maximum allowed number of attempts was 
reached without completing the interview. 

Response Status by Locating Characteristics 

Table 1 also shows the distribution of response 
status by whether or not the sampled persons 
provided both full name and address information in 
the NHES:91 interview. Nearly 90 percent (460 of 
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Table 1. — Number and percent of sampled persons, by response status and address information status 



Response status 


Total 


With address information 1 


Without address 
information 1 




Number 


Percent 


Number 


Percent 


Number 


Percent 


Total 


513 


loo.o 


460 


100.0 


53 


100.0 


Located 


452 


88.1 


408 


88.7 


44 


83.0 


Completed interviews 


409 


90.5 


374 


91.7 


35 


79.5 


Not completed interview 


43 2 


9.5 


34 


8.3 


9 


20.5 


Not located 


61 


11.9 


52 


11.3 


9 


17.0 



1 Having address information means the respondent provided both full name and address in the NHES:91 interview. 

^Of the 43 interviews that were not completed, 21 were refusals, 19 were not reached after multiple attempts, and 3 were not 
completed for other reasons. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey (NHES), 
followup survey, 1992. 



513) of the sampled persons provided this 
information. Both the ability to locate the sampled 
respondents and their willingness to complete the 
interview once located appear to be greater for 
those who gave full name and address information 
in the NHES:91 interview, but the sample size is 
too small to conclude this from this study. 



Another key locating characteristic is whether the 
sampled person was located at the same telephone 
number used in the NHES:91 interview. Table 2 
shows the response distribution of the sampled 
persons by whether or not they were located at the 
same telephone number in 1991 and 1992. Slightly 
more than three-fourths (77 percent) of the 



Table 2. — Number and percent of sampled persons, by response status and telephone location 



Response status 


Total 


At same telephone number 


Not at same telephone number 


Number 


Percent 


Number 


Percent 


Number 


Percent 


Total 


513 


100.0 


394 


100.0 


119 


100.0 


Located 


452 


88.1 


394 


100.0 


58 


48.7 


Completed interview 


409 


90.5 


357 


90.6 


52 


89.7 


Not completed interview 


43 


9.5 


37 


9.4 


6 


10.3 


Not located 


61 


11.9 


- 


- 


61 


51.3 



SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey (NHES), 
followup survey, 1992. 
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sampled persons were located by dialing the same 
telephone number. All of the persons who were 
never located are categorized as not being at the 
same telephone number based on the assumption 
that they would have been located if they still had 
the same number. Although some of them actually 
may have been at the same telephone number, it is 
likely that only a very few persons were in that 
category because repeated calls were made to the 
original number if no other telephone numbers 
were identified for the respondent. Of the sampled 
persons who were at the same telephone number, 
91 percent completed the followup interview. 
Only 44 percent of those not at the same telephone 
number completed the interview (52 out of 119). 
However, 90 percent of those not at the same 
telephone number who were located completed the 
interview. Thus, the difference in completion rates 
at this stage was the inability to locate the persons. 

Table 3 shows the distribution of response status 
by both the persons' willingness to provide name 
and address in the initial interview and by whether 
or not they were located at the same telephone 
number. The numbers are relatively small for 
these tabulations and show little of substance. 



Response Status by Tracing Efforts 

Locating the sampled respondent was the first and 
most important step in completing the followup 
interview. As shown in table 3, 119 of the 513 
sampled persons were not at the same telephone 
number and required some type of locating 
activity. For 113 of these persons, some type of 
locating effort could be attempted, while for 6 
persons nothing could be done because of the lack 
of information on even the name of the sampled 
person. Because of their importance, more detailed 
results of applying the locating procedures are 
discussed below. 

If the person was not located at the original 
number, the first possible step to locate a 
respondent involved recorded messages that 
provided new telephone numbers. Only 18 
telephone numbers had recordings with new 
telephone numbers: 13 of these respondents were 
located, 1 1 produced completed interviews, and 5 
were not located (not in table). 



Table 3. — Number of sampled persons, by response status, telephone location, and address 
information 



Response status 


At same telephone number 


Not at same telephone number 


With address 
information* 


Without address 
information* 


With address 
information* 


Without address 
information* 


Total 


359 


35 


101 


18 


Located 


359 


35 


49 


9 


Completed interview 


322 


35 


43 


9 


Not completed 


37 


0 


6 


0 


interview 










Not located 


0 


0 


52 


9 



* Having address information means the respondent provided both full name and address in the NHES:91 interview. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey (NHES), 
followup survey, 1992. 
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Before the results from the other locating activities 
are presented, some factors associated with the 
results must be considered. First, although the 
locating activities were done sequentially, the 
activity associated with ultimately locating the 
respondent cannot specifically be identified. For 
example, the Postal Service may have returned a 
postcard with a new address for a respondent after 
the credit searches for the person had begun. In 
this case, directory assistance for the person would 
have been attempted immediately using the new 
address. In another example, a source may have 
identified a telephone number that had been 
attempted previously. The number might have 
been temporarily disconnected or the respondent 
might have been away during the earlier calling 
period; when the number was dialed again, the 
respondent was located. It is unclear how to 
associate a source with locating the respondent in 
these and many other situations. 

Because of the difficulty of linking the source of 
the telephone number to locating and interviewing 
the respondent, it is not possible to fully assess the 
effectiveness of each source of locating 
information. Instead, the final results of locating 
and interviewing are presented for all the 



respondents who were subjected to the various 
types of locating activities. The discussion begins 
with the postcards returned by the Postal Service 
for respondents who were no longer at their 1991 
address and ends with the credit bureau search 
efforts. 

All 460 sampled persons who provided a name and 
address in the NHES:91 were mailed a postcard 
(exhibit 1). Postcards were returned by the Postal 
Service for 59 of these persons (13 percent). Of 
the postcards that were returned, 44 contained new 
address information. Other people were 
successfully interviewed, although 61 percent were 
located. After dialing the original telephone 
number and using data from the returned postcard, 
directory assistance was dialed for those sampled 
persons who were still not located but had provided 
their name and address in the NHES:91 interview. 
As shown in Table 4, directory assistance was used 
for 97 cases (2 1 percent). Only 44 percent of those 
persons were located and 37 percent completed the 
followup interview. 

The results of directory assistance shown in table 4 
also include telephone calls made to contact 
persons whom respondents identified in the 



Table 4. — Number and percent of sampled persons, by response status and tracing activity 



Response status 



Tracing activity 


Total 


Located 


Located, completed 
interview 


Located, not 
completed 
interview 


Not located 


Number 


Number 


Percent 


Number 


Percent 


Number 


Percent 


Number 


Percent 


Postcard returns 


44 


27 


61.4 


23 


52.3 


4 


9.1 


17 


38.6 


Directory assistance/ 






- 














contact person 


97 


43 


44.3 


36 


37.1 


7 


7.2 


54 


55.7 


Credit searches 




















with new locating data 


15 


4 


26.7 


3 


20.0 


1 


6.7 


11 


73.3 


without new locating data 


33 


6 


18.2 


5 


15.2 


1 


3.0 


27 


81.8 



SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey (NHES), 
followup survey, 1992. 



NHES:91 interview. The results of these activities 
cannot be separated, because the telephone 
interviewers immediately attempted to reach the 
contact persons if no new telephone information 
was obtained from directory assistance for the 
respondent. Contact persons were named for two- 
thirds of the 513 sampled persons, but calls were 
only needed for 44 of these persons. Twenty-eight 
postcards returned by the Postal Service had notes 
such as "Address unknown." Table 4 shows the 
distribution of the response status for the 44 
persons for whom the postcard mailing resulted in 
obtaining a new address. Only 52 percent of these 
calls resulted in a new telephone number to contact 
the sampled respondent. Of the remaining 16 
contact persons called, 12 said that they would call 
back on the Westat toll-free number after they 
spoke to the sampled respondent but ultimately did 
not call back, and the other 4 refused to provide 
any new data. 

The final locating strategy was the use of a credit 
bureau search for the sampled persons who had not 
been located by any of the previous efforts. Forty- 
eight names were sent to the credit service, and 15 
were returned with different information than 
previous sources had provided. In many of the 33 
other cases, the credit searches resulted in address 
information that had already been obtained from 
one of our previous tracing efforts. Only 27 
percent of the sampled persons for whom new 
locating information was obtained from the credit 
searches were located, and 20 percent completed 
the interview. 

After all the locating activities were completed, 
final attempts were made to contact all unresolved 
cases. In most cases, these efforts involved dialing 
the NHES:91 telephone number again. If the 
persons had been located but not interviewed, the 
telephone number at which they were located was 
dialed again. These efforts resulted in some 
completed interviews. This is also the reason that 
five completed interviews are shown in table 4 for 
the cases in which the credit searches did not result 
in new locating data. Some persons located and 
interviewed in this process had telephones that had 
been temporarily disconnected, and others had 



been away from home during the initial field 
period. 

A summary of all the locating efforts attempted is 
presented as table 5. This table gives the number 
and percent of persons by whether or not they were 
located and interviewed for each of the different 
types of locating activities attempted. The locating 
efforts , were essential in improving the response 
rates to the survey. As a result of the locating 
activities, about 10 percent (53 of 513) of the 
respondents were located and 8 percent (43 of 513) 
were interviewed. The postcard returns and the 
directory assistance calls (including using the 
contact persons from the NHES:91 interview) were 
responsible for finding most of these respondents. 

These locating activities are relatively low cost 
efforts and should be included in any followup 
surveys. 

The credit bureau searches are more expensive and 
did not result in locating many more of the 
respondents. The utility of the credit bureau 
searches needs to be closely considered in future 
studies. The cost of conducting the searches must 
be weighed against a possible 1 to 3 percent 
increase in the response rate. The direct cost of 
credit bureau searches to locate a person's 
telephone number varies depending on the nature 
of the search. Typically, it costs about $3 to $10 
per searched name, with other costs associated with 
handling and recording the data retrieved from the 
service. For followup surveys where the loss due 
to nonresponse is a major problem, it may be 
useful to incur the credit bureau search costs even 
for a relatively small increase in response rates. As 
noted earlier, new technologies such as the Internet 
may provide other methods for locating persons at 
reasonable cost, but these were not in use in 1992 
when this survey was conducted. 

Implications for Longitudinal Analysis 

The findings discussed above provide a framework 
for evaluating the feasibility of conducting a 
followup telephone survey with parents of young 
children when the sample is selected using RDD 



Table 5. — Number and percent of sampled persons, by response status and all types of locating activities 



Response status 


Type of locating activity 


Number of 
persons 


Percent 




Postcard returned 


Directory 

assistance 


Credit searches 






Located 


none 


none 


none 


399 


88.3 




none 


called 


none 


20 


4.4 




none 


called 


new data 


3 


0.7 




none 


called 


no new data 


4 


0.9 




return 


none 


none 


8 


1.8 




return 


called 


none 


15 


3.3 




return 


called 


new data 


1 


0.2 




return 


called 


no new data 


2 


0.4 


All located 


- 


- 


- 


452 


100.0 


Located and completed 


none 


none 


none 


366 


89.5 




none 


called 


none 


16 


3.9 




none 


called 


new data 


2 


0.5 




none 


called 


no new data 


3. 


0.7 




return 


none 


none 


5 


1.2 




return 


called 


none 


14 


3.4 




return 


called 


new data 


1 


0.2 




return 


called 


no new data 


2 


0.5 


All located and completed 


- 


- 


- 


409 


100.0 


Located and not completed 


none 


none 


none 


33 


76.7 




none 


called 


none 


4 


9.3 




none 


called 


new data 


1 


2.3 




none 


called 


no new data 


1 


2.3 




return 


none 


none 


3 


7.0 




return 


called 


none 


1 


2.3 


All located and not 
completed 


- 


- 


- 


43 


100.0 


Not located 


none 


none 


none 


1 


1.6 




none 


none 


no new data 


1 


1.6 




none 


called 


none 


18 


29.5 




none 


called 


new data 


7 


11.5 




none 


called 


no new data 


17 


27.9 




return 


none 


none 


2 


3.3 




return 


called 


none 


2 


3.3 




return 


called 


new data 


4 


6.6 


- 


return 


called 


no new data 


9 


14.8 


All not located 


• 


- 


- 


61 


100.0 



NOTE: Percentages may not add to 100 because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey (NHES), 
followup survey, 1992. 
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methods. The two major issues that must be 
addressed in using the NHES for a longitudinal 
followup survey are coverage bias and unit 
nonresponse bias. 

Coverage bias arises because not all eligible 
persons live in households with telephones. 
Previous research on telephone coverage for young 
children indicated that the coverage bias in the 
estimates for this population will be relatively 
small if appropriate weighting adjustments are 
made during estimation (Brick et al. 1992a). Of 
course, estimates for domains that contain a 
disproportionately high fraction of the 
undercovered population may be subject to more 
severe biases. Unit nonresponse bias is the more 
immediate concern of this research. The size of the 
bias due to nonresponse is related to the percentage 
of the sample that did not respond to the survey 
and the differences in the characteristics of the 
respondents and the nonrespondents. One way of 
expressing this nonresponse bias as a fraction of 
the response rate and the difference in 
characteristics is: 

Bia S {y r ) = {\--{Y r -Y nr ) (1) 

n 

where y r is the estimate based on responses only 
from respondents, r is the number of respondents, 
n is the total sample size, Y r the value of the 
characteristics for respondents, and Y nr is the 
value of the characteristics for nonrespondents. 
This expression shows that as the response rate 

(— ) decreases and the difference between 

n 

respondent and nonrespondent characteristics 
increases, the bias of the estimate can become 
large. Kalton, Kasprzyk, and McMillen (1989) 
discuss the potential impact of nonresponse bias for 
longitudinal surveys. They point out the 
deleterious effect on the estimates as response rates 
decrease and differences between respondents and 
nonrespondents increase. 

The role of nonresponse in the bias of the estimates 
is the reason this experiment was developed to 
assess the potential rate in a longitudinal study. 



Differences in characteristics of respondents and 
nonrespondents are harder to quantify because the 
difference depends on both the specific items 
considered and the reasons for the nonresponse. 
For example, some characteristics, such as poverty, 
low maternal education, and high family mobility 
are educational risk factors for children (Zill et al. 
1995; Sattler 1992). To the extent factors such as 
these are associated with survey response and the 
ability to locate the subject during longitudinal 
followup, they may be sources of bias. 

Since no data were collected from nonrespondents 
in the followup, it is not possible to compare their 
characteristics with those of the respondents based 
on the followup interview. However, the initial 
NHES:91 interview was completed by all sampled 
cases, and these characteristics can be examined to 
provide some insight into the correlates of 
nonresponse in the followup. Of course, important 
characteristics such as moving to a different home 
and changing family composition are not included 
when using only the data from the initial interview. 

Data from the NHES:91 interviews were used to 
compute estimates of the differences between those 
who did and did not respond to the followup. The 
estimates were derived using the weights from the 
NHES:91 to account for the probability of selecting 
the persons for that survey. These estimates differ 
slightly from unweighted analysis. For example, 
the estimated response rate for the followup using 
the weights is 76 percent while the unweighted 
response rate is 80 percent. 

Table 6 presents estimates of the response rates to 
the followup by the demographic and early 
childhood characteristics from the NHES:91 
interview. Estimated differences between response 
rates to the followup are subject to very large 
sampling errors because the sample sizes are small. 
Thus, even moderately sized differences in 
response rates cannot be distinguished due to the 
large sampling errors. A few of the differences 
bear discussion because they are so large that they 



Table 6. — Followup response rates by child and household characteristics 



Characteristic 


Unweighted number of cases 1 


Weighted response rate 


Total 


512 


76 


Race 






White 


395 


85 


Black 


72 


49 


Other 


45 


80 


Hispanic origin 






Hispanic 


103 


72 


Non-Hispanic 


409 


79 


Home Type 






Single family or townhouse 


398 


84 


Apartment 


78 


56 


Other type 


36 


74 


Home ownership 






Own 


342 


86 


Rent 


152 


66 


Some other arrangement 


18 




Income 






$20,000 or less 


118 


68 


$20,001 to $40,000 


206 


78 


More than $40,000 


188 


88 


Parents’ highest education 2 






High school or less 


193 


1 72 ' 


Some college/vocational/technical 


160 


79 


Bachelor’s degree or more 


156 


85 


Mother’s first language 
English 


437 


78 


Spanish 


62 


75 


Other 


13 


__3 


Family receives daily newspaper 






Yes 


301 


81 


No 


210 


75 


Census region of residence 
Northeast 


95 


85 


Midwest 


128 


77 


South 


193 


75 


West 


96 


78 


Read to child in past week 






Yes 


463 


78 


No 


47 


80 


Child weighed 5.5 pounds or more at birth 






Yes 


477 


78 


No 


35 


80 



*One case is excluded from this analysis due to missing values. 

^Highest education of the child's parent(s) or guardian(s) living in the household. 

^ Estimates based on fewer than 30 cases (unweighted) are not shown and are indicated by 



SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey (NHES), 
followup survey, 1992. 
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are statistically significant despite the large 
sampling errors. 1 

The most important difference in followup 
response rates was between white and black 
persons. The estimated response rate was 49 
percent for blacks and 85 percent for whites. With 
such a large difference between the response rates 
of blacks and whites, even small differences in the 
characteristics of the respondents could result in 
large biases. Thus, the nonresponse bias for 
estimates by race could be very substantial. 

It is interesting to note that the estimates by 
Hispanic origin were not large or statistically 
significant. The estimated response rate for 
Hispanics was 72 percent as compared to 79 
percent for non-Hispanics. 

Another characteristic that was statistically 
significant was whether the person lived in an 
apartment or single family house. Those who lived 
in an apartment had an estimated followup 
response rate of 56 percent, while those who lived 
in a single family house had an estimated response 
rate of 84 percent. Similarly, persons who owned 
their homes had higher response rates than renters 
(86 percent versus 66 percent). This result is 
consistent with the hypothesis that those who are 
most likely to move, such as renters and apartment 
dwellers, are underestimated in longitudinal 
surveys due to nonresponse bias. Allen and Petroni 
(1994) report on research to account for movers 
explicitly using different nonresponse adjustment 
methods, although they were notable to identify 
any adjustments that reduced attrition bias in their 
study. 

Persons in households with incomes over $40,000 
annually (88 percent) and those in households with 
incomes of $20,000 or less (68 percent) are 
significantly different. Similarly, children whose 
parents' highest education is a bachelor's degree or 
more have higher response rates than those whose 



1 The sampling errors for the differences in response rates were 
calculated by multiplying the simple random sampling error 

{yj pq I n) by the square root of the mean design effect for 
theNHES:91 ECE component. 
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parents' highest education is high school or less (85 
percent versus 72 percent). Because education and 
income are associated with home ownership and, 
hence, with mobility, these differences are not 
surprising. 

Most of the selected variables of substantive 
importance for early childhood education (e.g., 
how often the child was read to by a parent, 
whether the child weighed more than 5.5 pounds at 
birth, or whether the child was read to in the past 
week) had relatively small differences in estimated 
response rates. Of course, if the sample size were 
larger, many of these smaller differences would be 
statistically significant if the observed differences 
remained. 



Longitudinal Response Rates 

The response rate for a longitudinal survey is the 
completion rate for the initial survey multiplied by 
the followup completion rate, assuming that only 
those who responded to the initial survey are 
followed. In the NHES application, the followup 
response rate is the product of the response rate for 
the NHES:91 ECE interview and the completion 
rate for the followup survey. The response rate for 
the NHES:91 ECE interview was 77 percent (the 
response rate to the ECE was the product of the 8 1 
percent screening completion rate and the 95 
percent completion rate for the ECE interview). 
Thus, the overall response rate for longitudinal 
analysis using the followup data is 62 percent (77 
percent for the original survey times 80 percent for 
the followup survey). 

It might be possible to increase the followup 
response rate by increasing the percentage of 
respondents who are located. Often, the greater the 
time between contacts, the greater the percentage 
of cases that are lost to the study. Burgess (1989) 
reviews some approaches to tracing respondents in 
longitudinal surveys. One strategy to potentially 
increase the locating rate would be to have 
semiannual or quarterly postcard mailings. This 
relatively low-cost approach would increase the 
likelihood of obtaining change-of-address 
information from the Postal Service in a timely 
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way and would also remind the respondent of the 
study. By contrast, interim telephone contacts 
between data collection phases would be 
considerably more costly, but may be no more 
effective. 

It is interesting to speculate on what the response 
rate might be if the followup completion rates were 
somewhat different. For example, if all the persons 
who were located in the followup survey (88 
percent) were successfully interviewed, the 
followup response rate would be 68 percent (77 
percent times 88 percent). This is only 6 percent 
greater than the observed rate. If the locating rate 
were raised from 88 percent to, say, 95 percent, and 
the same 90 percent response from located persons 
was obtained, the followup response rate would be 
65 percent (77 percent times 86 percent), an 
increase of only 4 percent. Future followup efforts 
should seek to maximize both the locating rate and 
the response rate, but the potential for increasing 
the overall response rate using these methods is 
limited. 

Clearly, the followup response rate is depressed by 
the 8 1 percent completion rate for the screening of 
households in the initial NHES:91. Several 
attempts have been made to increase this rate in the 
NHES, for example, by refielding cases that have 
had the maximum number of attempts in the study 
protocol, and research in this area continues. For 
example, in the NHES:96, an advance mailing was 
sent to all households for which an address could 
be obtained from a commercial vendor. However, 
attaining completion rates of 85 percent or higher 
in this type of general population survey is not yet 
feasible. Assuming the screening rate is 85 percent 
and all the other rates are as observed, the 
longitudinal response rate would be 65 percent (85 
percent times 95 percent times 80 percent). Once 
again, this is higher than the observed rate, but only 
marginally. Conversely, if design decisions for the 
study lead to lower screener response rates, the net 
response rate for a longitudinal study would be 
decreased. 

In essence, this analysis suggests that the 
longitudinal response rate for a RDD survey like 
the NHES would probably be between 62 and 65 



percent. A 70 percent rate appears to be the upper 
limit, unless better techniques for gaining 
participation are discovered and implemented. To 
understand how these response rates compare with 
the rates that might be attained using different 
modes of sampling and data collection, the 
response rates for longitudinal household and 
cohort surveys conducted by the Census Bureau 
and the Ohio State University are described below. 
Although these comparisons are useful, it is 
important to recognize that the demands placed on 
the respondents in these surveys vary significantly 
and may affect some of the results. 

SIPP Response Rates 

The Survey of Income and Program Participation 
(SIPP) is a longitudinal household survey 
conducted by the Bureau of the Census. Since it is 
an important household-based panel survey, SIPP 
provides an appropriate comparison to the NHES 
experiment on the feasibility of collecting 
longitudinal data. 

Several SIPP surveys have been conducted since 
data collection started in 1984. The survey consists 
of an initial interview and followup surveys 
conducted at 4-month intervals. These data 
collections are called waves. For example, if the 
wave 1 interview takes place in January, then the 
wave 2 interview should be conducted in May of 
the same year. More details on the design of SIPP 
and the data we used on response rates can be 
found in Jabine et al. (1990). 

The first wave of data collection in SIPP is always 
done by personal visit to the household. 
Subsequent data collection is done primarily by 
phone but may also be done in person. Followup 
data are not collected for households that do not 
participate in the wave 1 data collection. For 
comparative purposes, we will use the rates 
reported for the 1988 Panel of SIPP, (i.e., the 
survey that began data collection in February 
1988). The overall response rates for the eight 
waves of this survey are given in table 7. It should 
be recognized that the population of interest in 
SIPP is not restricted to households with young 
children. 
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Table 7. — Response rates from the 1988 panel of the Survey of Income and Program Participation 



Response Condition 


Response rate 


Responded to all 8 interviews 


67.0% 


Responded to all interviews prior to death 


1.5 


Responded to all interviews except when: 
in an institution 


0.7 


in the Armed Forces 


1.4 


abroad 


0.6 


Other panel members 


0.2 


Total 


71.4 



SOURCE: Lepkowski, Kalton, and Kasprzyk (1989) 

The response rates reported for this survey are 
much higher than those that could be expected 
using an RDD screening sample. The main 

difference is the completion rate for the initial 
survey. The wave 1 rate for SIPP was 93 percent, 
while the NHES:91 ECE rate was 77 percent. 
While the difference in the rates is largely the 
result of the different modes of sampling and data 
collection, the Census Bureau's response rates are 
generally higher than those achieved by other data 
collection organizations. 

With personal interviews there are more 
opportunities to obtain participation, including 
activities such as speaking with neighbors, and it is 
easier to demonstrate the importance of the 
sampled person's cooperation. The Bureau's early 
experiments with RDD samples had response rates 
that are more consistent with the NHES response 
rates (Biemer, Chapman, and Alexander 1985; Fitti 
1979). 

The completion rates for followup surveys are also 
greater in SIPP than in the NHES experiment. For 
example, 94 percent of the wave 1 respondents 
completed the wave 2 interview in SIPP, while 
only 80 percent of the initial NHES:91 sample 
completed the followup interview. This is largely a 
function of being able to locate the respondents. In 
the followup to the NHES:91, 90 percent of those 
located were interviewed. The lack of information 
(including full name and address) to locate the 
sample persons in a RDD survey is a major 
impediment to improving the completion rates for 
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the followup. The fact that the SIPP followup was 
conducted only 4 months after the initial interview 
while the NHES followup was a year later also 
affects this comparison. Since the NHES followup 
was conducted 1 year after the initial interview, it 
is not possible to evaluate the impact of the time 
between interviews on the rates. Alternative 
procedures, such as interim postcard mailings and 
telephone contacts discussed earlier could reduce 
the numbers of cases lost to the study, but also add 
costs for a large-scale effort. 

NLSY Response Rates 

The National Longitudinal Survey of Youth 
(NLSY) is a longitudinal survey sponsored by the 
U.S. Bureau of Labor Statistics, U.S. Department 
of Labor that is conducted by the U.S. Bureau of 
the Census and the National Opinion Research 
Center. The survey is directed by the Ohio State 
University. In the NLSY, interviews are conducted 
with a sample of persons bom January 1, 1957 
through December 31, 1964. The base year for the 
survey was 1979. Annual followup surveys are 
conducted with this cohort. 

Table 8, below, gives the response rates for the 
NLSY civilian sample^ for the years 1979 through 



1 The NLSY included a civilian sample, a supplemental civilian 
sample, and a militaiy sample. Also, in 1986, a child sample 
was selected from among the children of women in the 
civilian samples. For ease of presentation, the civilian sample 
figures are presented here. Retention rates were similar for 
the supplemental civilian sample. 
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Table. 8. — Response and sample retention rates for the civilian sample of the National Longitudinal Survey of 
Youth: 1979 to 1991 



Survey year 


Interviews 

completed 


Percent of sample 
retained 


Response rate 


1979 (Base year) 


6111 


- 


81.8 


1980 


5873 


96.1 


78.6 


1981 


5892 


96.4 


78.8 


1982 


5876 


96.2 


78.6 


1983 


5902 


96.6 


79.0 


1984 


5814 


95.1 


77.7 


1985 


5751 


94.1 


76.9 


1986 


5633 


92.2 


75.4 


1987 


5538 


90.6 


74.1 


1988 


5513 


90.2 


73.7 


1989 


5571 


91.2 


74.6 


1990 


5498 


90.0 


73.6 


1991 


5556 


90.9 


74.3 



NOTE: The base year response rate is the product of the 91.2 percent screener completion rate and the 89.7 base year interview 
completion rate for the civilian sample. Retention rates reflect the percent of completed base year respondents who were interviewed 
the subsequent years, excluding ineligibles. Yearly response rates reflect the sample retention rate multiplied by the base year 
response rate. 

SOURCE: Center for Human Resources Research, Ohio State University. NLS User's Guide: 1993. Tables 1.1.1 and 1.1.3. 



1991. The retention rate for each followup phase 
multiplied by the base year response rate gives the 
response rate. The initial interviews were 
conducted in person, as were annual followup 
interviews through 1986. At that time, a change 
was made to a shorter interview administered by 
telephone. By the time the shift to telephone 
interviewing began, eight annual cycles of 
interviews had been conducted in person. At each 
followup point, efforts were made to interview all 
persons who completed the base year interview, 
whether or not the person had responded to the 
previous year's followup interview. 

The NLSY started with a base year response rate of 
82 percent in 1979. Over the 12 years from 1979 
through 1991, the response rate declined slightly, 
and the 1991 response rate was 74 percent. 







The NLSY response rates are higher than the 
NHES rate because of the higher base year 
response rate and the higher locating rate. One 
notable difference between the NHES and the 
NLSY is in the screening rate; the NLSY screener 
completion rate was 10 percent higher than the 
NHES:91 screening response rate. It is not 
uncommon for personal interview surveys to have 
higher initial response rates (Drew, Choudry, and 
Hunter 1988). The higher base year rate is 
essential to obtaining the higher response rates in 
the followup surveys. In addition, the NLSY has a 
higher completion rate at the followup stage than 
the NHES. Like the screening rate, this is probably 
related to the fact that the base year interview was a 
personal interview, so that address information was 
available for all cases at the base year data 
collection. In addition, in-person interviews tend 
to establish greater rapport and may have led to 
greater willingness to continue to participate in the 
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study. The NLSY, unlike the SIPP and NHES, 
uses a small incentive ($10) upon completion of 
the interview. 



Conclusions and Implications 

The results of the efforts to locate and interview the 
respondents for the NHES:91 longitudinal 
experiment indicate that there are significant 
questions about the feasibility of conducting a 
longitudinal survey using an RDD sample for the 
baseline. The main problem is nonresponse bias. 
The response rate for the base year of the ECE 
interview for the NHES:91 was 77 percent, and 
compounded with the locating and interviewing 
completion rates for the followup survey, the 
overall response rate was about 62 percent. Biases 
in surveys with rates this low are of great concern. 

Booth and Johnson (1985), who started with a 
lower base year response rate but obtained similar 
locating and interviewing rates, conclude that 
"panel studies based on random-digit-dialing 
samples can yield samples comparable to those 
gathered by more traditional methods." The 
authors of this report take a more cautious view. 
Although some longitudinal surveys with response 
rates even lower than the NHES feasibility study 
are conducted, alternative modes of data collection 
should be fully studied before using the NHES for 
this type of data collection. 

Although the nonresponse rate is due to both the 
initial nonresponse and the nonresponse at the 
followup, bias for the followup may be a more 
serious concern. The nonrespondents for the 
followup were largely persons who have moved (or 
at least changed telephone numbers) in the last 
year. This group of more mobile households may 
be very different from the responding households, 
which are less mobile. This could result in larger 
biases, especially because residential mobility is a 
risk factor for young children. Furthermore, the 
response rates for the followup were different for 
some important groups. In particular, the response 
rate for blacks was very low and could result in 
large biases by race. 



Some changes in the technology of telephone 
surveys may influence decisions about future 
surveys. Methods for locating persons based only 
on their telephone numbers may become available 
and would help to increase the rate at which 
persons can be located, although this change alone 
may not be sufficient for making the longitudinal 
approach viable in the NHES. If the initial 
screening response rates can also be significantly 
improved, then followup surveys using the NHES 
as the first wave should be reconsidered. 



Cost Considerations 

When evaluating the desirability of implementing a 
longitudinal study, cost factors must also be 
considered. The feasibility study was not designed 
to provide cost estimates for a full-scale study. A 
very brief and simple interview was used, which 
required a limited investment in the development 
of the CATI system and used only experienced 
interviewers. The costs associated with a longer 
and more complex instrument would be greater, 
particularly if skip patterns or edits based on 
responses from previous interviews were included. 
While this followup study used a fairly standard 
protocol for data collection, full-scale NHES 
collections typically include more elaborate refusal 
conversion methods and the refielding of cases that 
have reached the maximum number of attempts 
called for in the study protocol. 

In comparison with a base year random-digit-dial 
survey, a followup survey does not incur the 
substantial costs associated with screening large 
numbers of telephone numbers to identify 
household numbers and eligible household 
members. A longitudinal study would, however, 
incur substantial costs associated with conducting 
tracing activities to maximize the percentage of the 
sample that is located. Some of these activities, as 
noted previously, are relatively inexpensive. For 
example, mailing a postcard with "address 
correction requested" was found to be useful and 
inexpensive. Making directory assistance calls 
when the original telephone number is no longer 
valid is also a low cost method, as is making calls 
to contact persons named by the respondent in the 
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original interview. More costly tracing efforts, 
such as credit bureau checks, can be productive. 
However, it is most cost effective to schedule the 
followup field period such that other, less costly 
methods are exhausted before the cost of credit 
bureau checks is incurred. As mentioned, new 
technologies for locating persons may be useful in 
this regard. 

Burgess (1989) suggests that several rounds of 
tracing efforts conducted periodically between 
survey cycles may ultimately be more cost- 
effective than extensive (and perhaps less 
successful) tracing efforts at the followup survey 
point. While Burgess' example is for a multi-year 
time frame, this may also be germane to a 1-year 
followup as in the NHES feasibility study. The 
locating rate might be increased by relatively low- 
cost interim tracing activities, such as mailing a 
postcard with "address correction requested" about 
half-way between the initial survey and the 
followup point. 

The potential cost of longitudinal studies 
developed from the NHES could vary widely. 
These costs are dependent on the specific design 
elements of the study and the strategies required to 
implement those design elements. While it is very 
likely that the costs would be much smaller than 
surveys that are conducted in person, it is difficult 
to quantify the cost savings associated with this 
approach. 

Alternative Assessment of Change Over Time 

This research into a longitudinal analysis does not 
imply that the NHES cannot be used to precisely 
measure change over time in the population. 
Rather, it addresses longitudinal approaches that 
require contacting the same sampled persons 
repeatedly. Other ways of measuring change in the 
population are feasible. For example, changes in 
the characteristics of the population can be 
measured with independent cross-sectional 
surveys. With such a sample design, the variance 
of the estimated measure of change is the sum of 
the variances for the estimates at the two points in 
time. The variances of estimates of change are 



thus almost always larger than would be the case if 
the same persons were followed and interviewed. 

Alternative methods for reducing the variance of 
estimates of change can be employed without using 
the same sample of persons over time. The 
precision of estimates of change can be improved if 
the samples from the two points overlap. Since the 
overlapping sampled units tend to have large 
positive correlations for most characteristics of 
interest, the precision of estimates of change can be 
improved by this design. 

One way of inducing this correlation is to reuse the 
sampled clusters from the first stage of the RDD 
sample. These are the banks of 100 consecutive 
telephone numbers from which additional 
telephone numbers are generated using the 
standard or modified Waksberg methods of RDD 
sampling (see Brick and Waksberg (1991) for more 
details on these RDD designs). An open question 
is the size of the correlations that would be 
obtained using this method. If the correlations 
were relatively small, then the precision of the 
estimates would not be significantly improved. 
While the size of these correlations currently is 
unknown, it is likely to be small for the NHES 
population of young children because this group is 
relatively rare within a cluster. In addition, this 
method is not consistent with the move to list- 
assisted methods of RDD sampling (Casady and 
Lepkowski 1993; Brick et al. 1995) in the NHES, 
although the list-assisted method could be modified 
to achieve many of the same benefits. 

Another approach that might provide higher 
correlations and improved precision for estimates 
of change is to reuse the sampled telephone 
numbers. In this design, sampled respondents are 
not followed from year to year, but many of the 
people (about three-fourths based on the followup 
results) will be at the same telephone number. The 
number of persons who are sampled in both 
surveys will be less than this because of changes in 
household composition. 

This approach has two sample components: a 

sample of the telephone numbers dialed in the 
original survey, and a new sample of telephone 
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numbers. The new sample is needed to eliminate 
the bias associated with persons currently in 
households that were not valid telephone numbers 
when the original sample was selected. The 
number of households sampled from the two 
components is allocated based on the desired 
precision of the estimates. If estimates of change 
are more important than the cross-sectional 
estimates, more of the sample will be allocated to 
the numbers dialed in the original survey. Some 
additional information would also have to be 
obtained from the households to make national 
estimates. 

This approach improves the precision for estimates 
of change, but it is not a pure longitudinal sample 
design in which the sampled persons are repeatedly 
interviewed. It does avoid the major nonresponse 
problem of being unable to locate the sampled 



persons 1 year later. The other reasons for 
nonresponse will still affect respondents in this 
design. In particular, concerns about respondents 
becoming reluctant to respond to the survey after 
they are repeatedly interviewed still need to be 
evaluated. Given the two NHES goals of having 
high response rates and improving estimates of 
change, the reuse of the sampled telephone 
numbers is an approach that could be tested 
empirically. A subset of the sampled telephone 
numbers could be included in a future NHES 
survey, and the response rates for the newly 
sampled and repeatedly sampled numbers could be 
computed. This experiment would also provide 
data on the size of the expected correlations and the 
differences in the expected costs of data collection 
for the two treatments. This method of using the 
NHES to monitor changes over time appears more 
promising than retaining the same sampled persons 
over time. 
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